Task execution on a graphics processor using indirect argument buffers

ABSTRACT

The disclosure pertains to techniques for operation of graphics systems and task execution on a graphics processor. One such technique comprises a computer-implemented method for task execution on a graphics processor, the method comprising creating a data structure for grouping data resources, populating the data structure with two or more data resources for encoding into a graphics processing language by an encoding object, passing the data structure to a first programming interface command, the first programming interface command configured to access the data structure&#39;s data resources, triggering execution of a first function on a graphics processor in response to passing the data structure to the first programming interface command, passing the data structure to a second programming interface command, the second programming interface command configured to access the data structure&#39;s data resources, and triggering execution of a second function on the graphics processor in response to passing the data structure to the second programming interface command.

BACKGROUND

The subject matter disclosed herein relate to the field of graphicsprocessing and, without limitation, techniques for task execution on agraphics processor using indirect argument buffers.

Graphics processing units (GPUs) have become important for processingdata-parallel graphics tasks. Developers now recognize that non-graphicsdata-parallel tasks can also be handled by GPUs, taking advantage oftheir massive parallel capabilities. Vendors and standards organizationshave created application programming interfaces (APIs) that makegraphics data-parallel tasks easier to program. However, there are alsolow-level APIs (or libraries/frameworks etc.) that reside closer tohardware and are generally employed by applying the output ofhigher-level APIs. In other words, the higher-level APIs generallyprepare program code for application to the lower-level APIs.

To take advantage of certain GPU capabilities, it may be necessary topass a set of resources to the GPU via multiple API calls. Each API callhas a non-insignificant overhead cost associated with it. Additionally,where a particular set of resources are used from frame to frame,passing this set of resources repeatedly for multiple API calls overpossibly multiple frames may be resource inefficient and time consuming.

SUMMARY

This disclosure relates generally to the field of computer programming.More particularly, but not by way of limitation, aspects of the presentdisclosure relates to a computer-implemented method for task executionon a graphics processor, the method comprising creating a data structurefor grouping data resources, populating the data structure with two ormore data resources for encoding into a graphics processing language byan encoding object, passing the data structure to a first programminginterface command, the first programming interface command configured toaccess the data structure's data resources, triggering execution of afirst function on a graphics processor in response to passing the datastructure to the first programming interface command, passing the datastructure to a second programming interface command, the secondprogramming interface command configured to access the data structure'sdata resources, and triggering execution of a second function on thegraphics processor in response to passing the data structure to thesecond programming interface command.

Another aspect of the present disclosure relates to acomputer-implemented method for task execution on a graphics processor,the method comprising receiving a request to encode a data structureinto a graphics processing language, the data structure for grouping twoor more data resources, the request having an indication that the datastructure may be re-indexed, determining whether to re-index the datastructure based on one or more characteristics of the graphicsprocessor, encoding the data resources into an allocated memory for thedata structure based on the determination, receiving a first call for afirst programming interface command, the first call including the datastructure, executing a first function on the graphics processor inresponse to the first call, wherein executing the first functionincludes accessing the data structure's data resources, receiving asecond call for a second programming interface command, the second callincluding the data structure, and executing a second function on thegraphics processor in response to the second call, wherein executing thesecond function includes accessing the data structure's data resourcesreceiving a request to create a data structure for grouping dataresources.

Another aspect of the present disclosure relates to a non-transitoryprogram storage device, readable by a processor and comprisinginstructions stored thereon to cause one or more processors to create adata structure for grouping data resources, populate the data structurewith two or more data resources for encoding into a graphics processinglanguage by an encoding object, pass the data structure to a firstprogramming interface command, the first programming interface commandconfigured to access the data structure's data resources, triggerexecution of a first function on a graphics processor in response topassing the data structure to the first programming interface command,pass the data structure to a second programming interface command, thesecond programming interface command configured to access the datastructure's data resources, and trigger execution of a second functionon the graphics processor in response to passing the data structure tothe second programming interface command.

Another aspect of the present disclosure relates to a non-transitoryprogram storage device, readable by a processor and comprisinginstructions stored thereon to cause one or more processors to receive arequest to encode a data structure into a graphics processing language,the data structure for grouping two or more data resources, the requesthaving an indication that the data structure may be re-indexed,determine whether to re-index the data structure based on one or morecharacteristics of the graphics processor, encode the data resourcesinto an allocated memory for the data structure based on thedetermination, receive a first call for a first programming interfacecommand, the first call including the data structure, execute a firstfunction on the graphics processor in response to the first call,wherein executing the first function includes accessing the datastructure's data resources, receive second call for a second programminginterface command, the second call including the data structure; andexecute a second function on the graphics processor in response to thesecond call, wherein executing the second function includes accessingthe data structure's data resources.

The architecture implied by these embodiments provides a level ofindirection between an application's use of graphics and the way thegraphics are actually rendered. Therefore, the aforementionedembodiments allow an application program (and its developer) todisregard the graphics backend. This has many advantages. For example,by managing the graphics backend independently, any improvements made inthe backend may accrue to application programs without any change to theapplication code. Thus, if there is new improved hardware and improvedlow-level libraries to take advantage of the hardware, under someembodiments, even old applications might take advantage of new features.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a representative computer node that may be used, forexample, as an end-user machine or a developer machine.

FIG. 2 shows an illustrative network environment associated with one ormore embodiments.

FIG. 3 shows an illustrative software layer and architecture diagram.

FIG. 4 shows an illustrative graphics system.

FIGS. 5A, 5B, and 5C illustrate an indirect argument buffer and usagethereof, according to an embodiment.

FIG. 6 is a block diagram illustrating re-indexing, according to anembodiment.

FIG. 7 is a flow diagram illustrating a technique for utilizing indirectargument buffers, according to an embodiment.

FIG. 8 is a flow diagram illustrating a technique for encoding indirectargument buffers, according to another embodiment.

DETAILED DESCRIPTION

This disclosure pertains to systems, methods, and computer readablemedia to improve the operation of graphics systems. More specifically,aspects of the present disclosure relates to task execution on agraphics processor. Task execution on programmable pipelines of agraphics processor generally includes various inputs, such as thosesetting up a task for execution, along with the inputs to that task. Forexample, information related to a rendering task may includeinstructions for the rendering itself, information related to the shapesbeing rendered, textures overlay the shapes, lighting information, etc.Information may be grouped into a set of information and input into thegraphics processor from a higher level program using a using a singledata structure to pass the set of information to the graphics processor.This set of information may be stored in a location accessible by bothapplication code executing on a central processing unit (CPU), as wellas by code executing on a graphical processing unit (GPU). This datastructure may also be reused for multiple function calls.

In the following description, for purposes of explanation, numerousspecific details are set forth in order to provide a thoroughunderstanding of the disclosed concepts. As part of this description,some of this disclosure's drawings represent structures and devices inblock diagram form in order to avoid obscuring the novel aspects of thedisclosed concepts. In the interest of clarity, not all features of anactual implementation are described. Moreover, the language used in thisdisclosure has been principally selected for readability andinstructional purposes, and may not have been selected to delineate orcircumscribe the claimed subject matter, leaving resorting to the claimsas a potential necessity to determine such claimed subject matter.Reference in this disclosure to “one embodiment” or to “an embodiment”or “embodiments” means that a particular feature, structure, orcharacteristic described in connection with the embodiment is includedin at least one embodiment of the disclosed subject matter, and multiplereferences to “one embodiment” or “an embodiment” should not beunderstood as necessarily all referring to the same embodiment.

It will be appreciated that in the development of any actualimplementation (as in any software and/or hardware development project),numerous decisions must be made to achieve the developers' specificgoals (e.g., compliance with system- and business-related constraints),and that these goals may vary from one implementation to another. Itwill also be appreciated that such development efforts might be complexand time-consuming, but would nonetheless be a routine undertaking forthose having the benefit of this disclosure and being of ordinary skillin the design and implementation of graphical processor interfacesoftware or graphical processing systems.

Exemplary Hardware and Software

The embodiments described herein may have implication and use in andwith respect to all types of devices, including single- andmulti-processor computing systems and vertical devices (e.g., cameras,gaming systems, appliances, etc.) that incorporate single- ormulti-processing computing systems. The discussion herein is made withreference to a common computing configuration that may be discussed as asoftware development system or an end-user system. This common computingconfiguration may have a CPU resource including one or moremicroprocessors. This discussion is only for illustration regardingsample embodiments and is not intended to confine the application of theclaimed subject matter to the disclosed hardware. Other systems havingother known or common hardware configurations (now or in the future) arefully contemplated and expected. With that caveat, a typical hardwareand software operating environment is discussed below. The hardwareconfiguration may be found, for example, in a server, a workstation, alaptop, a tablet, a desktop computer, a gaming platform (whether or notportable), a television, an entertainment system, a smart phone, aphone, or any other computing device, whether mobile or stationary.

Referring to FIG. 1, the disclosed embodiments may be performed byrepresentative computer system 100. For example the representativecomputer system may act as a software development platform or anend-user device. System 100 may be embodied in any type of device suchas a general purpose computer system, a television, a set top box, amedia player, a multi-media entertainment system, an image processingworkstation, a hand-held device, or any device that may be coupled withor may incorporate display or presentation devices as discussed herein.Computer system 100 may include one or more processors 105, memory 110(110A and 110B), one or more storage devices 115, and graphics hardware120. Computer system 100 may also have device sensors 125, which mayinclude one or more of: depth sensors (such as a depth camera), 3D depthsensor(s), imaging devices (such as a fixed and/or video-capable imagecapture unit), RGB sensors, proximity sensors, ambient light sensors,accelerometers, gyroscopes, any type of still or video camera, LIDARdevices, SONAR devices, microphones, CCDs (or other image sensors),infrared sensors, thermometers, etc. These and other sensors may work incombination with one or more GPUs, DSPs or conventional microprocessorsalong with appropriate programming so the sensor outputs may be properlyinterpreted and/or combined and interpreted.

Returning again to FIG. 1, system 100 may also include communicationinterface 130, user interface adapter 135, and display adapter 140—allof which may be coupled via system bus, backplane or communicationfabric 145, which may comprise one or more switches or one or morecontinuous (as shown) or discontinuous communication links. Memory 110may include one or more different types of media (e.g., solid-state,DRAM, optical, magnetic, etc.) used by processor 105 and graphicshardware 120. For example, memory 110 may include memory cache,read-only memory (ROM), and/or random access memory (RAM). Storage 115may include one or more non-transitory storage media including, forexample, magnetic disks (fixed, floppy, and removable) and tape, opticalmedia such as CD-ROMs and digital video disks (DVDs), and semiconductormemory devices such as Electrically Programmable Read-Only Memory(EPROM), and Electrically Erasable Programmable Read-Only Memory(EEPROM). Memory 110 and storage 115 may be used to retain media (e.g.,audio, image, and video files), preference information, device profileinformation, computer program code or instructions organized into one ormore modules and written in any desired computer programming language,and any other suitable data. When executed by processor 105 and/orgraphics hardware 120, such computer program code or instructions mayimplement one or more of the methods or processes described herein.Communication interface 130 may include semiconductor-based circuits andbe used to connect computer system 100 to one or more networks.Illustrative networks include, but are not limited to: a local network,such as a USB network; a business's local area network; and a wide areanetwork such as the Internet and may use any suitable technology (e.g.,wired or wireless). Communications technologies that may be implementedinclude cell-based communications (e.g., LTE, CDMA, GSM, HSDPA, etc.) orother communications (Ethernet, WiFi, Bluetooth, USB, Thunderbolt,Firewire, etc.). User interface adapter 135 may be used to connectkeyboard 150, microphone 155, pointer device 160, speaker 165, and otheruser interface devices such as a touchpad and/or a touch screen (notshown). Display adapter 140 may be used to connect one or more displayunits 170.

Processor 105 may execute instructions necessary to carry out or controlthe operation of many functions performed by system 100 (e.g.,evaluation, transformation, and compilation of graphics programs).Processor 105 may, for instance, drive display 170 and receive userinput from user interface adapter 135 or any other user interfacesembodied by the system. User interface 135, for example, can take avariety of forms, such as a button, a keypad, a dial, a click wheel, akeyboard, a display screen, and/or a touch screen. Processor 105 may beany type of computing device such as one or more microprocessors workingalone or in combination with GPUs, DSPs, system-on-chip devices such asthose found in mobile devices. Processor 105 may include one or morededicated GPUs or graphics subsystems that accept program instructionsto create or alter display information such as pixels. In addition,processor 105 may be based on reduced instruction-set computer (RISC) orcomplex instruction-set computer (CISC) architectures or any othersuitable architecture and may include one or more processing cores.Graphics hardware 120 may be special purpose computational hardware forprocessing graphics and/or assisting processor 105 in performingcomputational tasks. In some embodiments, graphics hardware 120 mayinclude CPU-integrated graphics and/or one or more programmable GPUs.System 100 (implementing one or more embodiments discussed herein) canprovide the means for one or more users to control the same system(e.g., system 100) or another system (e.g., another computer orentertainment system) through user activity, which may include naturalactivity and/or predetermined gestures such as hand gestures.

Various embodiments of the disclosed subject matter may employ sensors,such as cameras. Cameras and like sensor systems may include auto-focussystems to accurately capture video or image data ultimately used tointerpret user intent or commands. Since the motion of the user may bebased upon subtle activity in small regions in the captured images(e.g., hands, fingers, face, mouth, brow etc.) the autofocus system maybe used to separately focus on multiple regions of the image in order toaccess better information. Returning to FIG. 1, sensors 125 may capturecontextual and/or environmental phenomena such as time; locationinformation; the status of the device with respect to light, gravity,and the magnetic north; and even still and video images. In addition,network-accessible information such as weather information may also beused as part of the context. All captured contextual and environmentalphenomena may be used to provide context to user activity or informationabout user activity. For example, in accessing a gesture or theexpression or emotion of a user, the contextual information may be usedas part of the analysis. If the time is 3:00 a.m., it is more likelythat a user's face indicates sleepiness than sadness.

FIG. 2 depicts illustrative network architecture 200, within which thedisclosed techniques may be implemented and the disclosed hardware mayreside. This illustrative network 200 may include a plurality ofnetworks 205, (i.e., 205A, 205B, and 205C), each of which may take anyform including, but not limited to, a local area network (LAN) or a widearea network (WAN), such as the Internet. Further, networks 205 may useany desired technology (wired, wireless, or a combination thereof) andprotocol (e.g., transmission control protocol, TCP). Coupled to networks205 are data server computers 210 (i.e., 210A and 210B) that are capableof operating server applications such as databases and also capable ofcommunicating over networks 205. One embodiment using server computersmay involve the operation of one or more central systems to processgraphics information and distribute the processed information to nodeson a network.

Client computers 215 (i.e., 215A, 215B, and 215C), which may take theform of any smartphone, gaming system, tablet, computer, set top box,entertainment device/system, television, telephone, communicationsdevice, or intelligent machine, including embedded systems, may also becoupled to networks 205, and/or data server computers 210. In someembodiments, network architecture 210 may also include network printerssuch as printer 220 and storage systems such as 225, which may be usedto store multi-media items or other data that are referenced herein. Tofacilitate communication between different network devices (e.g., dataservers 210, end-user computers 215, network printer 220, and storagesystem 225), at least one gateway or router 230 may be optionallycoupled there between. Furthermore, in order to facilitate suchcommunication, each device employing the network may comprise a networkadapter circuit and related software. For example, if an Ethernetnetwork is desired for communication, each participating device musthave an Ethernet adapter or embedded Ethernet-capable ICs. Further, thedevices may carry network adapters for any network in which they mightparticipate (including, but not limited to, PANs, LANs, WANs, andcellular networks).

As noted above, embodiments of the subject matter disclosed hereininclude software. As such, a description of common computing softwarearchitecture is provided as expressed in a layer diagram in FIG. 3. Likethe hardware examples, the software architecture discussed here is notintended to be exclusive in any way, but rather to be illustrative. Thisis especially true for layer-type diagrams, which software developerstend to express in somewhat differing ways. In this case, thedescription begins with the base hardware layer 395 illustratinghardware 340, which may include CPUs and GPUs or other processing and/orcomputer hardware. Above the hardware layer is the O/S kernel layer 390showing an example O/S kernel 345, which is kernel software that mayperform memory management, device management, and system calls (oftenthe purview of hardware drivers). The notation employed here isgenerally intended to imply that software elements shown in a layer useresources from the layers below and provide services to layers above.However, in practice, all components of a particular software elementmay not behave entirely in that manner.

Returning to FIG. 3, layer 385 is the O/S services layer exemplified byO/S services 350. O/S services may provide core O/S functions in aprotected environment. In addition, O/S services shown in layer 385 mayinclude frameworks for OpenGL® 351 (OpenGL is a registered trademarkowned by Silicon Graphics Inc.), Metal® 352 (Metal is a registeredtrademark owned by Apple Inc.), Software Raytracer 353, and a PureSoftware Rasterizer 354. These particular examples all relate tographics and/or graphics libraries and are chosen to illuminate thetopic of many embodiments herein which relate to graphics handling.These particular examples also represent graphics frameworks/librariesthat may operate in the lower tier of frameworks, such that developersmay use shading and graphics primitives and/or obtain fairly tightlycoupled control over the graphics hardware. In addition, the particularexamples named in FIG. 3 may also pass their work product on to hardwareor hardware drivers.

Referring again to FIG. 3, OpenGL 351 represents an example of awell-known library and application-programming interface (API) forgraphics rendering including 2D and 3D graphics. Metal 352 alsorepresents a published graphics library and framework, but it is lowerlevel than OpenGL 351, supporting fine-grained, low-level control of theorganization, processing, and submission of graphics and computationcommands, as well as the management of associated data and resources forthose commands. Software Raytracer 353 is software for creating imageinformation based upon the process of tracing the path of light throughpixels in the plane of an image. Pure Software Rasterizer 354 refersgenerally to software used to make graphics information such as pixelswithout specialized graphics hardware (e.g., using only the CPU). Theselibraries or frameworks shown within the O/S services layer 385 are onlyexemplary and intended to show the general level of the layer and how itrelates to other software in a sample arrangement (e.g., kerneloperations usually below and higher-level Applications Services 360usually above). In addition, it may be useful to note that Metal 352represents a published framework/library of Apple Inc. that is known todevelopers in the art. Furthermore, OpenGL 351 may represent aframework/library present in current versions of software distributed byApple Inc.

Above the O/S services layer 385 there is an Application Services layer380, which includes SpriteKit® 361, SceneKit® 362, Core Animation® 363,and Core Graphics 364 (SpriteKit, SceneKit, and Core Animation areregistered trademarks owned by Apple Inc.). The O/S services layerrepresents higher-level frameworks that are commonly directly accessedby application programs. In some embodiments, the O/S services layerincludes graphics-related frameworks that are high level in that theyare agnostic to the underlying graphics libraries (such as thosediscussed with respect to layer 385). In such embodiments, thesehigher-level graphics frameworks are meant to provide developer accessto graphics functionality in a more user/developer friendly way andallow developers to avoid work with shading and graphics primitives. Byway of example, SpriteKit 361 is a graphics rendering and animationinfrastructure made available by Apple Inc. SpriteKit 361 may be used toanimate textured images or “sprites.” SceneKit 362 is a 3D-renderingframework from Apple Inc. that supports the import, manipulation, andrendering of 3D assets at a higher level than frameworks having similarcapabilities, such as OpenGL. Core Animation 363 is a graphics renderingand animation infrastructure made available from Apple Inc. CoreAnimation 363 may be used to animate views and other visual elements ofan application. Core Graphics 364 is a two-dimensional drawing enginefrom Apple Inc. Core Graphics 365 provides 2D rendering forapplications.

Above the application services layer 380, there is the application layer375, which may comprise any type of application program. By way ofexample, FIG. 3 shows three specific applications: photos 371 (a photomanagement, editing, and sharing program), a Quicken® brand financialmanagement program 372 (Quicken is a registered trademark owned byIntuit Inc.), and an iMovie® brand movie making and sharing program 373(iMovie is a registered trademark owned by Apple Inc.). Applicationlayer 375 also shows two generic applications 370 and 374, whichrepresent the presence of any other applications that may interact withor be part of the embodiments disclosed herein. An application program,such as application 370, may call into one or more API, service, orframework to display content. Generally, embodiments of the disclosedsubject matter employ and/or interact with applications to producedisplayable/viewable content.

In evaluating O/S services layer 385 and applications services layer380, it may be useful to realize that different frameworks have higher-or lower-level application program interfaces, even if the frameworksare represented in the same layer of the FIG. 3 diagram. Theillustration of FIG. 3 serves to provide a general guideline and tointroduce exemplary frameworks that may be discussed later. Furthermore,some embodiments may imply that frameworks in layer 380 make use of thelibraries represented in layer 385. Thus, FIG. 3 provides intellectualreinforcement for these examples. Importantly, FIG. 3 is not intended tolimit the types of frameworks or libraries that may be used in anyparticular way or in any particular embodiment.

Referring to FIG. 4, the disclosed embodiments may be performed byrepresentative graphics system 400. For example, representative graphicssystem 400 may act to process application data and render graphicalrepresentations of virtual objects to a display 402. For example, a CPU404 may receive a request from application code (not shown) to render agraphic. The request may be via a graphics library and framework, suchas Metal 352 or OpenGL 351. The graphic may be a portion of a model of avirtual object comprising one or more polygons, such as a triangle. Thisrequest may reference data stored, for example, in memory 406 or videomemory 408. The CPU 404 may communicate via bus, switch, or fabric 410with GPU 420. The GPU 420 may include a graphical pipeline including oneor more vertex shaders 422, one or more rasterizers 424, and one or morefragment shaders 426. In some embodiments, a unified memory model may besupported where memory 406 and video memory 408 comprise a single memoryutilized by both the GPU 420 and CPU 404 rather than discrete memorysystems. As used herein, application code may refer to code executing onCPU 404 during application run time, separate from graphical functions,which may execute on GPU 420. Hardware components of GPU 420, such asshaders, may be programmable, allowing for graphical functions, such asshaders, to execute on GPU 420. API and Driver software, executing onCPU 404 may facilitate interactions between application code andgraphical functions, such as by providing an interface betweenapplication code and GPU 420 and allowing the application code to set upand execute graphical functions on GPU 420.

Memory storage modes defining storage location and access permissionsmay be supported by both memory 406 and video memory 408. For example,where a discrete memory model is used, memory 406 may support a sharedaccess mode defining system memory accessible by both the GPU 420 andCPU 404. Video memory 408 may support a private access mode defining atleast a portion of video memory 408 as only accessible by the GPU 420.Additionally both memory 406 and video memory 408 may support a managedaccess mode defining a synchronized memory pair for a resource with onecopy of the resource in memory 406 and another copy of the resource invideo memory 408. As another example, where the unified memory model isused, the memory 406 may support private and shared access modes wherethe private mode defines system memory accessible only to the GPU 420and the shared mode defines system memory accessible by both the GPU 420and CPU 404.

Generally, the GPU 420 may render a view of a virtual object using thevirtual object's model coordinate system. The virtual object may berendered from the point of view of a camera at a specified location. Thevertex shaders 422 perform matrix operations on the coordinates of aparticular polygon to determine coordinates at which to render thepolygon from the point of view of the camera based on the modelcoordinates. The rasterizer 424 then determines which pixels of thedisplay are intersected by the polygon. The fragment shader 426 thenassigns a color value to each of the pixels intersected by the polygon.This color value may be based, for example, on a particular texture.This texture may be stored in memory 406 or video memory 408. Shaders422 and 426 may be programmable as a part of a programmable GPU pipelineusing shader functions to allow for increased flexibility andfunctionality of the shaders. This programmability also allows the GPUto perform non-graphical, data-parallel tasks. In certain embodiments,the rasterizer 424 may be a fixed function of the GPU pipeline to allowfor increased performance. After the polygon is shaded, the polygon maybe written to a frame buffer in video memory 406 for use by the display402.

According to certain examples, application code, or a higher layer, mayrequest a graphics framework to render a frame for display. Rendering aframe may require one or more rendering passes and multiple graphicalAPI function calls. Graphical API function calls may be used to setupand execute programmable graphical functions such as a rendering pass. Agraphical API function call may include information describing one ormore virtual objects for display. This description may include resourcesfor use by the rendering pipeline, such as one or more polygons, orprimitives, texture information, samples, as well as informationdefining the state of the rendering pipeline. For example, anapplication code may attempt to render a virtual object, such as a wall,using a set of vertices describing polygons, which make up an apparentstructure of the wall, along with textures which may be placed on thepolygons. In setting up the GPU for a render pass, the application codemay call a first API to pass polygon information, a second API to pass atexture, and a third API to pass a sampler for use by, for example, ashader function. These resources may then be copied by the graphicsdriver to a private memory area and used when the shader function isexecuted during a draw loop. Each API call has a non-insignificantoverhead cost associated with its API call. Additionally, where aparticular set of resources is used from frame to frame, passing thisset of resources repeatedly for multiple API calls over possiblymultiple frames may be resource inefficient and time consuming.

Indirect Argument Buffers

Referring to FIGS. 5A-5C, embodiments of the disclosure allow a set ofresources of different data types to be grouped and referenced togetherin an indirect argument buffer (IAB) 502. Rather than passing eachresource of a set of resources individually over multiple graphical APIcalls, the set of resources may be placed in the IAB 502, which allowsset of resources of multiple data types to be organized into a singledata structure which may be collectively sent to the GPU using a singlelightweight graphical API call. As an example, IAB 502 may be aspecialized buffer that holds an opaque representation of a set ofresources which may be assigned together in a graphical API call.Generally, a buffer may comprise a code object that represents anallocation of unformatted, accessible memory which can contain data. IAB502 may extend a basic buffer type as a composite data type which allowsa grouped list of mixed data types to be stored and referenced together.Application code may instantiate the IAB 502 and assign a set ofresources, such as textures, samplers, and buffers, to the IAB 502 foruse in one or more graphical API calls. This allows an application codedeveloper to better organize a set of resources for use by certaingraphical API calls. Additionally, the IAB 502 may be used for one ormore graphical API calls, rather than assigning the resources to eachgraphical API call individually.

The number, type, and size of data contained in the IAB 502 may bedefined by the application code. In certain embodiments, the IAB 502may, similar to other buffers, contain basic data types, such asfloating point and integers, vector data types, matrix data types,arrays of buffer types, structs of buffer types, and inlined constantdata. The IAB may also include data types such as pointers to otherIABs, textures, samplers, and arrays, both bound an unbound. The IAB maybe defined and populated outside of the draw loop. The application codemay have fine control over the logical layout of the resources withinthe IAB (but not the physical layout of the IAB). In this example, theIAB 502 was instantiated with a logical structure defining a set of datatype resources. These resources may be laid out in a specific order,here, starting with a texture 504, followed by a sample 506, and endingwith a pointer 508. IAB 502 may also more than one of a single data typesuch as additional textures 504, or samples 506. IABs may also referenceanother IAB. For example, IAB 502 may reference another IAB (not shown)using a pointer 508 to the other IAB. Pointers may also be utilized toreference other data types, such as other types of buffers. This set ofresources may then be passed together in a single API call forprocessing by, for example, a shader in a rendering pipeline. IAB 502may be passed via an API call A, for example, to graphical function A522 for execution in GPU pipeline state A 520 (see FIG. 5B). Inaccordance with this disclosure, the set of resources passed by API callA may be used again by another graphical function B 532. As shown inFIG. 5C the IAB 502 may also be passed via API call B for execution inGPU pipeline state B 530.

An IAB may be instantiated by application code using a particularlogical layout for one or more types of resources and, aftercompilation, passed to the GPU via an encoder. According to certainaspects, where this application code is compiled, a compiler, inconjunction with a driver for the GPU, may re-index (e.g., reorder) andchange the structure of the IAB such that the physical manifestation ofthe IAB is different from the logical structure of the IAB. Anindication of this re-indexing may be provided, such as flags or hints,which enables an encoder to lay out and encode the re-indexed IAB. Atrun time, the encoder may encode the IAB into a GPU language, such as ashader language, using the regular structure of the shading languagesuch that the IAB appears no different in the shading language thanother structures. This structural consistency allows, for example,shaders to reference specific parts of an IAB as if the IAB were aregular shader structure. Once encoded, the layout of both IAB 602 and622 may be fixed and no additional changes to the structure orre-indexing may occur. A person having ordinary skill in the art mayappreciate that a compiler may be used to transform application codeinto computer languages other than binary or object code, such as anintermediary representation without departing from aspects of thisdisclosure.

Generally, a texture is composed of pixel data and metadata related tothe pixel data. According to certain aspects, this metadata may beencoded in the IAB and the actual pixel data may be stored separately.This metadata may include, for example, texture state information (e.g.,dimensions, format, etc.). The metadata may also include a pointer toseparately stored pixel data. In other cases, the encoded IAB maycontain a pointer to the metadata, which may then point to separatelystored pixel data. In other cases, the encoded IAB may contain both themetadata and the actual pixel data.

FIG. 6 is a block diagram illustrating re-indexing 600, according to anembodiment. In certain cases, the compiler in conjunction with a GPUdriver may modify the physical layout of an IAB 602. This modificationmay be made to take into account physical characteristics of the GPU,such as the width of available data channels, GPU pipeline length, etc.For example, application code may organize an IAB 602 by declaring IAB602 having a first texture 604 with an index of 1, with an associatedfirst sampler 606 with an index of 2, and an associated first pointer608 with an index of 3, along with a second texture 610 with an index of4, an associated second sampler 612 with an index of 5, and anassociated second pointer 614 with an index of 6. During compilation ofthe application code, a compiler may indicate that IAB 602 may bere-indexed, for example, based on a set flag or configuration, or adetermination based on the application code being compiled. Thisre-indexing may help optimize GPU performance. For example, certain GPUsmay benefit from clustered data types. In such a case, at run time, thedriver may re-index the IAB 602 to arrange the data types in such anorder, such as shown in IAB 622, with a first texture 624 with an indexof 1, a second texture 630 with an index of 2, a first sampler 626 withan index of 3, a second sample 632 with an index of 4, a first pointer628 with an index of 5, and a second pointer 634 with an index of 6.This re-indexing may be transparent to the application code and thedriver or API may maintain a mapping, such as a mapping table, of there-indexing. The GPU function may then access the IAB 622 using there-indexed structure directly without referencing the original structureof IAB 602 or the mapping table. As the encoder object may be aware ofthe layout used by the driver and for different pipeline states, thedriver may index IAB 602 in exactly the same way each time. Theapplication code may continue to use the indexing of IAB 602 inreferencing specific resources in IAB 602 without being aware of there-indexing.

FIG. 7 is a flow diagram 700 illustrating a technique for indirectargument buffers according to an embodiment. At 702, application codemay create an IAB having two or more data objects of a supported type.The application code may also specify an index attribute n (for example,a 32-bit unsigned integer) that may be used to identify a specificresource within the IAB. The application code may receive an IABallocated in a shared memory space accessible by both the GPU and theCPU. Where a shared memory space is not utilized, the IAB may be createdin memory accessible to the CPU. At 704, after the IAB is created, theapplication code may populate the IAB memory space with resources. Thispopulation may be performed using an encoder object to write to the IAB.The encoder object may be used to write to the IAB as the applicationcode may not know the physical memory layout of the IAB. While theapplication code is able to access specific resources within the IAB,the driver may re-index the IAB as needed based on, for example,hardware limitations. Where shared memory space is not utilized, the IABmay be copied into a private memory space accessible only to the GPUupon encoding.

After the IAB is encoded, the driver may be made aware of the IAB via agraphical API call referencing the IAB. According to certain aspects,the graphical API call may both encode and reference the IAB. At 706,the application code may call a first API command referencing the IABand passing the IAB to the GPU. The GPU may then perform an actionassociated with the first API command, such as a shader function usingresources in the IAB. In certain cases, a particular graphical functionmay just use the resources that it needs. Not all the resources in theIAB may be used by the particular graphical function. The GPU may alsowrite data, such as graphical function outputs or results to the IAB. At708, the application code may call a second API command passing the IAB.The GPU may then perform an action associated with the second APIcommand using resources in the IAB; such resources could have beenplaced into the IAB during acts in accordance with block 704 or by theGPU itself during acts in accordance with block 706.

FIG. 8 is a flow diagram 800 illustrating a technique for encodingindirect argument buffers, according to another embodiment. At 802, acomputing system may receive a request to encode a data structure into agraphics processing language, the data structure for grouping two ormore data resources, the request having an indication that the datastructure may be re-indexed. At 804, the computing system may determinewhether to re-index the data structure based on one or morecharacteristics of the graphics processor. At 806, the computing systemmay encode resources to populate an IAB. At 808, the computing systemmay receive a first call to an API command passing in the IAB. At 810,the computing system may execute a first function associated with theAPI call on the GPU. The first function may access one or more resourcesin the IAB. At 812, the computing system may receive a second call to anAPI command passing in the IAB. At 814, the computing system may executea second function associated with the API call on the GPU. The secondfunction may also access one or more resources in the IAB.

As the IAB may be instantiated outside a draw loop, the overheadrequired to track the residency of the IAB resources may be reduced. Anobject associated with a resource may be considered resident when theobject is accessible by the GPU. Ordinarily, this occurs during thecreation of a graphical API object based on resources passed to the GPUvia a graphical API call. On graphical API object destruction, thegraphical API object may be removed and is no longer resident as theobject is no longer accessible to the GPU. The graphics driver generallytracks object residency on a per graphical API basis. For resourcesincluded in the IAB, residency of these resources may be established asa part of instantiating the IAB object once for each rendering passrather than multiple times over multiple API calls for a singlerendering pass. By establishing residency during instantiation, theapplication developer is able to control and track resource residencyand have better visibility regarding the resources used for a renderingor compute pass. Additionally, as the IAB may be created outside of thedraw loop, the set of resources may be encoded to shading languagearguments once rather than for each time the set of resources arerequired. The IAB may also be allocated from a heap to further reducethe residency cost.

According to certain aspects the GPU may modify the IAB, allowing forGPU driven pipelines and indirect or multi-draw indirect graphicalfunctions to be executed by the GPU. For example, an output of a shaderfunction may be written into the IAB and used as input for anothergraphical API call. As another example, an IAB may be encoded withoutresources and a graphical function may populate the IAB during runtimefor use by another graphical function. As an example, the limit on thenumber of resources that may be placed in an IAB may be set to a largernumber dynamically or set to a relatively large number by default or inresponse to an argument. Additionally, unbound arrays may be used in anIAB.

Generally, a graphical function executing on the GPU may modify IABs asneeded, including IABs referenced within a command buffer itself.However, application code executing on the CPU generally may not modifythe IAB safely between the time that the command buffer referencing theIAB is committed and the time the GPU has completed the command bufferas the application code is generally unable to determine the state ofgraphical functions during this time.

In certain cases, it may be known that neither the application code norgraphical functions will modify an IAB from the time that the IAB isassigned as a graphical API argument until the time the GPU hascompleted execution of the command buffer. In such cases, the IAB may beconsidered immutable. For example, a flag may be set when compiling orthe compiler may determine during compilation that an IAB is immutable.In such cases, the compiler and driver may be able to perform certainoptimizations and a hint may be added to a pipeline descriptor toidentify IABs that are mutable. This hint may be used by the GPU at runtime to optimize execution of a particular pipeline based on whether theIAB is mutable or immutable. In one embodiment, if this hint is notspecified, it may be assumed that IABs are immutable by default.

It is to be understood that the above description is intended to beillustrative, and not restrictive. The material has been presented toenable any person skilled in the art to make and use the disclosedsubject matter as claimed and is provided in the context of particularembodiments, variations of which will be readily apparent to thoseskilled in the art (e.g., many of the disclosed embodiments may be usedin combination with each other). In addition, it will be understood thatsome of the operations identified herein may be performed in differentorders. The scope of the disclosed subject matter, therefore, should bedetermined with reference to the appended claims, along with the fullscope of equivalents to which such claims are entitled. In the appendedclaims, the terms “including” and “in which” are used as theplain-English equivalents of the respective terms “comprising” and“wherein.”

The invention claimed is:
 1. A computer-implemented method for taskexecution on a graphics processor, the method comprising: creating adata structure for grouping data resources; populating the datastructure with two or more data resources for encoding into a graphicsprocessing language by an encoding object, wherein the data structure isordered based on an index associated with each data resource, andwherein the encoding object is configured to reorder the data structureby changing the index associated with at least one of the datastructure's data resources; passing the data structure to at least oneprogramming interface command, the at least one programming interfacecommand configured to access the data structure's data resources; andtriggering execution of least one function on the graphics processer inresponse to passing the data structure to the at least one programminginterface command.
 2. The computer-implemented method of claim 1,wherein the data structure is stored in a shared memory space accessibleby both a central processor and the graphics processor.
 3. Thecomputer-implemented method of claim 2, wherein the encoding object isconfigured to reorder the data structure based on a physicalcharacteristic of the graphics processor.
 4. The computer-implementedmethod of claim 1, further comprising modifying data in the datastructure based on processing the at least one programming interfacecommand.
 5. The computer-implemented method of claim 1, whereintriggering execution of the at least one function on the graphicsprocessor further comprises modifying, by the graphics processor, atleast some of the data structure's data resources.
 6. Thecomputer-implemented method of claim 1, wherein the two or more dataresources are of two or more data types.
 7. The computer-implementedmethod of claim 1, wherein the at least one programming interfacecommand is a first programming interface command, and wherein the atleast one function is a first function whose execution is triggered onthe graphics processor in response to passing the data structure to thefirst programming interface command, and wherein the method furthercomprises: passing the data structure to a second programming interfacecommand, the second programming interface command configured to accessthe data structure's data resources; and triggering execution of asecond function on the graphics processer in response to passing thedata structure to the second programming interface command.
 8. Thecomputer-implemented method of claim 7, wherein the first function isdifferent from the second function.
 9. The computer-implemented methodof claim 7, wherein the first function is configured to modify at leastone of the data structure's data resources to create a modified datastructure, and wherein the modified data structure is passed to thesecond programming interface command.
 10. A non-transitory programstorage device, readable by a processor and comprising instructionsstored thereon to cause one or more processors to: create a datastructure for grouping data resources; populate the data structure withtwo or more data resources for encoding into a graphics processinglanguage by an encoding object, wherein the data structure is orderedbased on an index associated with each data resource, and wherein theencoding object is configured to reorder the data structure by changingthe index associated with at least one of the data structure's dataresources; pass the data structure to at least one programming interfacecommand, the at least one programming interface command configured toaccess the data structure's data resources; and trigger execution of atleast one function on a graphics processer in response to passing thedata structure to the at least one programming interface command. 11.The non-transitory program storage device of claim 10, wherein the datastructure is stored in a shared memory space accessible by both acentral processor and the graphics processor.
 12. The non-transitoryprogram storage device of claim 10, wherein the encoding object isconfigured to reorder the data structure based on a physicalcharacteristic of the graphics processor.
 13. The non-transitory programstorage device of claim 10, further comprising instructions storedthereon to cause the one or more processors to modify data in the datastructure based on processing the at least one programming interfacecommand.
 14. The non-transitory program storage device of claim 10,wherein triggering execution of the at least one function on thegraphics processor further comprises modifying, by the graphicsprocessor, at least some of the data structure's data resources.
 15. Thenon-transitory program storage device of claim 10, wherein the at leastone programming interface command is a first programming interfacecommand, and wherein the at least one function is a first function whoseexecution is triggered on the graphics processor in response to passingthe data structure to the first programming interface command, andwherein the non-transitory program storage device further comprisesinstructions stored thereon to cause the one or more processors to: passthe data structure to a second programming interface command, the secondprogramming interface command configured to access the data structure'sdata resources; and trigger execution of a second function on thegraphics processer in response to passing the data structure to thesecond programming interface command.
 16. The non-transitory programstorage device of claim 15, wherein the first function is different fromthe second function.
 17. The non-transitory program storage device ofclaim 15, wherein the first function is configured to modify at leastone of the data structure's data resources to create a modified datastructure, and wherein the instructions cause the one or more processorsto pass the modified data structure to the second programming interfacecommand.
 18. A computer system comprising: a graphics processor; memory;one or more processors operatively coupled to the memory and thegraphics processor, the memory comprising instructions that cause theone or more processors to: create a data structure for grouping dataresources; populate the data structure with two or more data resourcesfor encoding into a graphics processing language by an encoding object,wherein the data structure is ordered based on an index associated witheach data resource, and wherein the encoding object is configured toreorder the data structure by changing the index associated with atleast one of the data structure's data resources; pass the datastructure to at least one programming interface command, the at leastone programming interface command configured to access the datastructure's data resources; and trigger execution of at least onefunction on the graphics processer in response to passing the datastructure to the at least one programming interface command.
 19. Thecomputer system of claim 18, wherein the encoding object is configuredto reorder the data structure based on a physical characteristic of thegraphics processor.
 20. The computer system of claim 18, wherein the atleast one programming interface command is a first programming interfacecommand, and wherein the at least one function is a first function whoseexecution is triggered on the graphics processor in response to passingthe data structure to the first programming interface command, andwherein the memory further comprises instructions that cause the one ormore processors to: pass the data structure to a second programminginterface command, the second programming interface command configuredto access the data structure's data resources; and trigger execution ofa second function on the graphics processer in response to passing thedata structure to the second programming interface command.
 21. Thecomputer system of claim 20, wherein the first function is configured tomodify at least one of the data structure's data resources to create amodified data structure, and wherein the modified data structure ispassed to the second programming interface command.